Wire JSON ingestion schema extension modules#1215
Conversation
3436470 to
ea857eb
Compare
b80bef4 to
bce83e3
Compare
ea857eb to
7af1788
Compare
bce83e3 to
38d7488
Compare
38d7488 to
65c8468
Compare
7af1788 to
20ee5dd
Compare
65c8468 to
d780944
Compare
5b67eda to
28d3b58
Compare
08fa741 to
66a50ff
Compare
4f85649 to
fd4651f
Compare
e904b45 to
7cd0f7d
Compare
2e3770f to
9ed0d47
Compare
3d879aa to
01d66b2
Compare
9ed0d47 to
a02256c
Compare
01d66b2 to
4d549ec
Compare
a02256c to
180c7d3
Compare
180c7d3 to
cd6256d
Compare
myronmarston
left a comment
There was a problem hiding this comment.
I haven't finished reviewing but wanted to submit my feedback so far.
55175c5 to
16255f2
Compare
myronmarston
left a comment
There was a problem hiding this comment.
Still not done reviewing but here's my next round of feedback.
738964b to
57c341a
Compare
myronmarston
left a comment
There was a problem hiding this comment.
Next set of feedback (still not done reviewing!).
a1e2bb3 to
e28ff9c
Compare
e28ff9c to
83f6c2a
Compare
myronmarston
left a comment
There was a problem hiding this comment.
Made it through the end of my review this time! Almost ready.
| schema_results = json_ingestion_schema_definition_results | ||
|
|
||
| @json_schemas_artifact ||= new_yaml_artifact( | ||
| JSON_SCHEMAS_FILE, | ||
| JSONSchemaPruner.prune(schema_results.current_public_json_schema), |
There was a problem hiding this comment.
| schema_results = json_ingestion_schema_definition_results | |
| @json_schemas_artifact ||= new_yaml_artifact( | |
| JSON_SCHEMAS_FILE, | |
| JSONSchemaPruner.prune(schema_results.current_public_json_schema), | |
| @json_schemas_artifact ||= new_yaml_artifact( | |
| JSON_SCHEMAS_FILE, | |
| JSONSchemaPruner.prune(json_ingestion_schema_definition_results.current_public_json_schema), |
Since it's memoized it's nice to limit things before the ||=.
| base_artifacts = super | ||
|
|
||
| versioned_artifacts = build_desired_versioned_json_schemas(json_schemas_artifact.desired_contents).values.map do |versioned_schema| | ||
| new_versioned_json_schema_artifact(versioned_schema) | ||
| end | ||
|
|
||
| base_artifacts + [json_schemas_artifact] + versioned_artifacts |
There was a problem hiding this comment.
| base_artifacts = super | |
| versioned_artifacts = build_desired_versioned_json_schemas(json_schemas_artifact.desired_contents).values.map do |versioned_schema| | |
| new_versioned_json_schema_artifact(versioned_schema) | |
| end | |
| base_artifacts + [json_schemas_artifact] + versioned_artifacts | |
| versioned_artifacts = build_desired_versioned_json_schemas(json_schemas_artifact.desired_contents).values.map do |versioned_schema| | |
| new_versioned_json_schema_artifact(versioned_schema) | |
| end | |
| super + [json_schemas_artifact] + versioned_artifacts |
| EOS | ||
| end | ||
|
|
||
| def format_deprecated_elements(deprecated_elements) |
There was a problem hiding this comment.
We need to be careful here--there's the potential for a naming conflict. If two EG extension libraries both defined an extension module for SchemaArtifactManager (say, elasticgraph-json_ingestion and elasticgraph-proto-ingestion), and they both defined a method named format_deprecated_elements, they could run into a collision. Each extension would work correctly in isolation but would potentially be buggy when both extensions were used.
That's unlikely to happen for any method that has json_schema in the name...but format_deprecated_elements, missing_field_error_for, missing_type_error_for, missing_necessary_field_error_for, and files_noun_phrase have pretty generic names--I can imagine any extension defining methods with those names and colliding.
This is similar to my comment from last review. Ideally the methods defined on an extension module all fall into one of two buckets:
- They offer a new public API
- They hook into/override existing methods to customize behavior
I'm not as worried about collisions in these cases. Public APIs are thoughtfully named and unlikely to collide; hooking into existing methods is performed in a composable way with super.
There are two general approaches I'd recommend here:
- Instead of an extension module, wrap with a
DelegateClassclass, like we did with elasticgraph-json_ingestion/lib/elastic_graph/json_ingestion/schema_definition/indexing/field_type/enum.rb in Add JSON ingestion indexing extensions #1204. However, it won't work here because wrapping doesn't let you hook in and overrideartifacts_from_schema_def(there's no way to intercept that via a wrapping class because it's an internal private method). - Keep the extension module for the methods which fit the criteria above. For everything else, move it into a separate class or module which you delegate to.
I think the 2nd option will work best here but the 1st option may come in handy in other cases.
(Tangentially, I have a similar concern about your results_extension.rb module as well--as it defines so many private methods----but there all the methods have json_schema in the name so a naming collision is less likely. If you get a hankering to give it the same treatment, feel free, but also fine to leave it!).
| module ScalarTypeExtension: ::ElasticGraph::SchemaDefinition::SchemaElements::ScalarType | ||
| include HasJSONSchema |
There was a problem hiding this comment.
| module ScalarTypeExtension: ::ElasticGraph::SchemaDefinition::SchemaElements::ScalarType | |
| include HasJSONSchema | |
| module ScalarTypeExtension: ::ElasticGraph::SchemaDefinition::SchemaElements::ScalarType | |
| include HasJSONSchema |
| add_filter "/elasticgraph-local/" unless spec_files_to_run.any? { |f| f.include?("/elasticgraph-local/") } | ||
|
|
||
| # The JSON ingestion gem is being introduced by extracting implementation first and moving its tests later. | ||
| add_filter "/elasticgraph-json_ingestion/" |
There was a problem hiding this comment.
Have you removed this in a follow up PR? (If so can you link it)?
Why
Introduce the JSON ingestion schema-definition extension modules after the indexing extension points exist, but before making JSON ingestion the default implementation.
What
ElasticGraph::JSONIngestion::SchemaDefinition::APIExtensionand supporting factory/results/artifact/state/schema-element extension modulesRisk Assessment
Medium - this adds new extension code, but the default behavior remains the existing core JSON Schema implementation in this PR.
References
bundle exec rspec elasticgraph-schema_definition/spec/unit/elastic_graph/schema_definition/json_schema_spec.rb elasticgraph-schema_definition/spec/unit/elastic_graph/schema_definition/json_schema_field_metadata_spec.rb elasticgraph-schema_definition/spec/unit/elastic_graph/schema_definition/indexing/json_schema_with_metadata_spec.rb elasticgraph-schema_definition/spec/unit/elastic_graph/schema_definition/factory_spec.rbpassed.script/type_checkpassed.script/lintpassed.Stack
Current PR is marked with
->.